Plan: Add selectable libghostty-vt renderer backend

Goal

Implement @coder/libghostty-vt-node as an optional semantic renderer backend for agent-tty, selectable through:

• global CLI flag: --renderer <ghostty-web|libghostty-vt>
• environment variable: AGENT_TTY_RENDERER
• config default: defaultRenderer

The new backend should use libghostty-vt for terminal state semantics (snapshot, wait, getVisibleText) while preserving the existing ghostty-web + Playwright path for PNG screenshots and WebM export fallback.

Verified context and constraints

• Current renderer contract is RendererBackend in src/renderer/backend.ts with boot, replayTo, snapshot, screenshot, getVisibleText, dispose, rendererBackend, and isBooted.
• Current default implementation is GhosttyWebBackend in src/renderer/ghosttyWeb/backend.ts.
• Current hardcoded construction points:
  • src/host/hostMain.ts creates HostRendererManager with new GhosttyWebBackend(...).
  • src/replay/offlineReplay.ts defaults offline replay to new GhosttyWebBackend(...).
  • src/export/webm.ts defaults WebM export to new GhosttyWebBackend(...).
• HostRendererManager in src/host/renderer.ts already accepts a backend factory, but it is currently effectively fixed per host instance and keyed around profile changes rather than renderer-name changes.
• Current env var convention is confirmed as AGENT_TTY_*; use AGENT_TTY_RENDERER, not AGENT_TERMINAL_RENDERER.
• @coder/libghostty-vt-node@0.1.0-beta.0 currently exports:
  • createTerminal({ cols, rows, scrollbackLimit? })
  • terminal methods: feed, resize, snapshot, getVisibleText, optional formatPlain, optional formatHtml, dispose
  • snapshot fields: cols, rows, cursorRow, cursorCol, isAltScreen, visibleLines, optional scrollbackLines, optional cells
  • getNativeInfo()
• The native package is semantic only. It does not render screenshots, video, or GUI output.

Advisor review refinements

An advisor review confirmed the plan direction and recommended these refinements, which are incorporated below:

• Prefer per-command renderer selection over purely session-scoped selection. CLI/env/config provide defaults, but render-affecting RPC and offline replay calls should carry the selected renderer name.
• Keep HostRendererManager minimal with one live backend at a time, keyed by (rendererName, profile). Recreate on renderer/profile changes; do not introduce a backend pool in v1.
• Do not persist renderer as session identity in v1. Treat session/env/config renderer values as defaults, not immutable ownership.
• Keep libghostty-vt semantic-only in v1. Use it for snapshot, getVisibleText, and render waits; use ghostty-web for PNG/WebM fallback.
• Preserve the distinction between requested renderer and actual artifact producer. rendererBackend should remain the backend that actually created the artifact; a fallback screenshot requested with libghostty-vt should still report ghostty-web as the PNG producer.

Phase 1 — Renderer selection plumbing

1. Add renderer registry

Create src/renderer/registry.ts with:

• RendererNameSchema = z.enum(['ghostty-web', 'libghostty-vt'])
• type RendererName = z.infer<typeof RendererNameSchema>
• DEFAULT_RENDERER_NAME = 'ghostty-web'
• resolveRendererName(input: string | undefined): RendererName
• createRendererBackend(rendererName, sessionId, profile, options?)

Implementation notes:

• Use existing assertion/schema style; unknown renderer names should throw a clear validation error.
• The registry should construct:
  • GhosttyWebBackend for ghostty-web
  • LibghosttyVtBackend for libghostty-vt
• Keep any optional test hooks local and explicit, e.g. backend factory overrides for tests.

2. Add CLI/env/config resolution

Update:

• src/cli/main.ts
  • add global .option('--renderer <name>', 'Default renderer backend name')
  • in the existing preAction hook, set process.env.AGENT_TTY_RENDERER after context resolution, mirroring AGENT_TTY_HOME / AGENT_TTY_LOG_LEVEL propagation.
• src/cli/context.ts
  • add renderer?: string to GlobalCliOptions
  • add rendererDefault: RendererName or string to CommandContext
  • resolve precedence as: CLI flag → AGENT_TTY_RENDERER → config defaultRenderer → ghostty-web
  • validate through resolveRendererName.
• src/config/resolveConfig.ts
  • add defaultRenderer: z.string().optional() to ConfigFileSchema.

Quality gate after Phase 1:

npm run typecheck
npm run test -- context resolveConfig renderer

Phase 2 — Make host/offline replay renderer-aware

1. Update host renderer manager selection

Update src/host/renderer.ts so HostRendererManager can select by renderer and profile.

Minimal approach:

• Change the backend factory shape to include renderer name:

type RendererBackendFactory = (
  rendererName: RendererName,
  sessionId: string,
  profile: RenderProfileConfig,
) => RendererBackend;

• Change getBackend(...) to accept rendererName:

getBackend(
  rendererName: RendererName,
  profile: RenderProfileConfig,
  replayInput: ReplayInput | null,
): Promise<RendererBackend>

• Cache only one active backend at a time, but include rendererName in the cache key along with the profile identity/hash. If either changes, dispose the old backend and create a new one.
  • This keeps the change minimal and avoids multiple live browser/native instances.
  • If concurrent mixed-renderer calls become a problem later, switch to a small map keyed by (rendererName, profileHash).

Defensive checks:

• Assert the selected renderer name has already been validated.
• Preserve existing replay/no-rewind/profile-size invariants.
• Ensure backend disposal remains idempotent.

2. Thread renderer through live RPC requests

Update internal protocol schemas in src/protocol/schemas.ts for render-affecting RPC params:

• SnapshotParamsSchema: add optional rendererName
• ScreenshotParamsSchema: add optional rendererName in addition to existing profile
• WaitForRenderParamsSchema: add optional rendererName

Then update callers/handlers:

• src/cli/commands/snapshot.ts
  • include rendererName: context.rendererDefault in RPC params and offline replay options.
• src/cli/commands/screenshot.ts
  • include rendererName: context.rendererDefault in RPC params and offline replay options.
  • keep --profile behavior unchanged; profile and renderer are separate.
• src/cli/commands/wait.ts
  • include rendererName: context.rendererDefault in waitForRender RPC params and offline replay options when render wait is used.
• src/host/hostMain.ts
  • resolve renderer per request using params.rendererName ?? process.env.AGENT_TTY_RENDERER ?? DEFAULT_RENDERER_NAME.
  • pass renderer name into rendererManager.getBackend(rendererName, profile, replayInput).
• src/replay/offlineReplay.ts
  • accept rendererName?: RendererName in options/deps.
  • create the selected backend through the registry when no test backendFactory override is provided.
• src/export/webm.ts and src/cli/commands/record-export.ts
  • for WebM export with rendererName === 'libghostty-vt', use the hybrid backend’s video fallback or explicitly resolve to the existing ghostty-web video-capable backend.
  • Do not claim native video support.

Quality gate after Phase 2:

npm run typecheck
npm run test -- host/renderer offlineReplay snapshot screenshot wait

Phase 3 — Add LibghosttyVtBackend

Create:

• src/renderer/libghosttyVt/backend.ts
• src/renderer/libghosttyVt/index.ts

Export it from src/renderer/index.ts if that file is the project’s public renderer export surface.

Backend behavior

LibghosttyVtBackend should implement RendererBackend only in v1. Do not make it implement VideoCapableRendererBackend initially; WebM/export should explicitly use ghostty-web while the selected semantic renderer remains libghostty-vt.

Core fields:

• readonly rendererBackend = 'libghostty-vt'
• isBooted: boolean
• native terminal handle from @coder/libghostty-vt-node
• last replay input / replay state, used for screenshot fallback
• lazy fallback GhosttyWebBackend for screenshot/video-only operations

Constructor:

constructor(
  sessionId: string,
  profile: RenderProfileConfig,
  options?: {
    loadNative?: () => Promise<typeof import('@coder/libghostty-vt-node')>;
    fallbackFactory?: (
      sessionId: string,
      profile: RenderProfileConfig,
      videoOptions?: VideoRecordingOptions,
    ) => GhosttyWebBackend;
  },
)

The options are for tests and should not leak into the public CLI surface.

Native loading

• Lazy import @coder/libghostty-vt-node in boot().
• If import or native loading fails, throw an actionable error:
  • selected renderer: libghostty-vt
  • package: @coder/libghostty-vt-node
  • recovery: install/update the package or use --renderer ghostty-web
• Call getNativeInfo() at debug level if available so dogfood logs can record the native package and Ghostty commit.

Replay mapping

In replayTo(input: ReplayInput):

• Assert the backend is booted and terminal exists.
• Preserve existing replay ordering/no-rewind behavior.
• Feed output events into the native terminal exactly as bytes according to the existing event schema used by GhosttyWebBackend.
• Apply resize events with terminal.resize(cols, rows).
• Ignore events that do not affect terminal screen state, but continue tracking sequence numbers.
• Store the latest ReplayInput or enough replay state to lazily replay into screenshot fallback.
• Return ReplayState consistent with current backend expectations: lastSeq, cols, rows, cursor position when available.

Defensive checks:

• Assert positive cols/rows before creating/resizing the native terminal.
• Assert no terminal method is called after dispose().
• Assert native snapshot dimensions match the replay/profile dimensions after replay unless a resize event intentionally changed them.

Snapshot mapping

Map @coder/libghostty-vt-node TerminalSnapshot into this repo’s SemanticSnapshot:

• visible lines: direct { row, text } mapping
• scrollback lines: include only when requested
• cells: include only when requested; map style fields conservatively
• cursor: map cursorRow / cursorCol
• alt screen: map isAltScreen
• dimensions: native cols / rows
• captured sequence: current replay state lastSeq
• session id: backend sessionId

Do not silently coerce missing or malformed native data. Validate through existing schemas where practical.

Screenshot and WebM fallback

screenshot(outputPath, options):

• Lazily create a GhosttyWebBackend with the same session/profile.
• Boot it if needed.
• Replay the last replay input into it.
• Call its screenshot(outputPath, options).
• Return the fallback result as-is so screenshot metadata honestly reports rendererBackend: 'ghostty-web' because ghostty-web produced the PNG.

WebM/export in v1:

• Do not add native video methods to LibghosttyVtBackend.
• Make record export --format webm --renderer libghostty-vt explicitly resolve the export backend to ghostty-web.
• Document that libghostty-vt is a semantic renderer, not a video renderer.

dispose():

• Dispose native terminal idempotently.
• Dispose fallback GhosttyWebBackend if created.
• Clear local handles and mark isBooted = false.

Quality gate after Phase 3:

npm run typecheck
npm run test -- libghosttyVtBackend backend-selection

Artifact metadata guidance

• Keep existing rendererBackend semantics as actual producer metadata.
  • Native snapshots should report libghostty-vt where the result shape supports backend metadata.
  • Screenshot fallback should report ghostty-web because the PNG was created by ghostty-web.
  • WebM export should report ghostty-web for the same reason.
• If reviewers later need auditability for fallback decisions, add a separate optional requestedRenderer field in a follow-up schema migration. Do not overload rendererBackend in this change.

Phase 4 — Dependency and packaging

Update package.json:

• Add @coder/libghostty-vt-node as an optionalDependency, pinned initially to 0.1.0-beta.0 or the version selected by the implementer.
• Keep imports lazy so unsupported platforms can still use the default renderer.
• Ensure npm run build does not require the native addon to load.

Update package/release docs only if needed:

• Document that libghostty-vt is optional/experimental.
• Document fallback behavior for screenshots/WebM.
• Avoid changing public skill examples unless there is a user-facing renderer example worth adding.

Quality gate after Phase 4:

npm install # or mise run bootstrap, depending on repo workflow
npm run build
npm run typecheck

Phase 5 — Tests

Unit tests

Add/extend:

• test/unit/renderer/registry.test.ts
  • default resolves to ghostty-web
  • valid names resolve
  • invalid names fail with useful message
• test/unit/renderer/libghosttyVtBackend.test.ts
  • lazy native loader is not called before boot()
  • boot() creates terminal with positive dimensions
  • replayTo() feeds output and resize events
  • snapshot() maps visible lines, scrollback, cursor, alt-screen, and cells
  • getVisibleText() delegates to native terminal
  • screenshot fallback replays into GhosttyWebBackend and returns PNG metadata
  • dispose() is idempotent
  • using methods after failed boot/dispose throws clearly
• test/unit/host/renderer.test.ts
  • manager recreates backend when renderer name changes
  • manager still recreates backend when profile changes
• test/unit/commands/* or existing command tests
  • RPC payloads include rendererName from context
  • offline replay receives selected renderer name

Use constructor dependency injection/mocks rather than loading the real native addon in unit tests.

Integration tests

Add/extend:

• test/integration/backend-selection.test.ts
  • --renderer ghostty-web preserves current behavior
  • AGENT_TTY_RENDERER=ghostty-web preserves current behavior
  • config defaultRenderer is honored
  • invalid renderer fails before spawning/using a backend
  • a mocked or test-only backend proves per-command renderer selection reaches running host RPC paths

For real native integration, use one guarded test block:

• Try importing @coder/libghostty-vt-node.
• If unavailable, skip with an explicit message.
• If available, run snapshot and wait --text against a simple fixture with AGENT_TTY_RENDERER=libghostty-vt.

E2E tests

Add a focused E2E test only if the native package loads in CI/dev:

• fixture: test/fixtures/apps/hello-prompt or equivalent
• flow: run → wait for prompt → type text → wait for echo → snapshot → screenshot
• assert:
  • snapshot/wait text includes expected prompt/echo
  • screenshot command succeeds through fallback
  • screenshot PNG exists and has nonzero size
  • artifact metadata distinguishes semantic backend versus screenshot fallback where the existing result shape supports it

Full validation before handoff:

mise run format-check
mise run lint
mise run typecheck
mise run test
mise run build

Fallback if mise is unavailable:

npm run format:check
npm run lint
npm run typecheck
npm run test
npm run build

Phase 6 — Dogfooding and reviewer proof

Create a proof bundle, for example:

dogfood/20260424-libghostty-vt-renderer/
  README.md
  commands.sh
  environment.txt
  native-info.json
  ghostty-web-snapshot.json
  libghostty-vt-snapshot.json
  libghostty-vt-wait.json
  libghostty-vt-screenshot.json
  ghostty-web-screenshot.json
  inspect.json
  screenshots/
    ghostty-web.png
    libghostty-vt-fallback.png
  recordings/
    terminal-session.cast
  videos/
    libghostty-vt-fallback.webm

Dogfood setup:

export AGENT_TTY_HOME="$(mktemp -d)"
npx tsx src/cli/main.ts --home "$AGENT_TTY_HOME" doctor --json

Baseline renderer run:

SESSION_WEB=$(npx tsx src/cli/main.ts --home "$AGENT_TTY_HOME" --renderer ghostty-web run --json -- npx tsx test/fixtures/apps/hello-prompt/main.ts | jq -r '.result.sessionId')
npx tsx src/cli/main.ts --home "$AGENT_TTY_HOME" --renderer ghostty-web wait "$SESSION_WEB" --text "READY>" --json
npx tsx src/cli/main.ts --home "$AGENT_TTY_HOME" --renderer ghostty-web snapshot "$SESSION_WEB" --json
npx tsx src/cli/main.ts --home "$AGENT_TTY_HOME" --renderer ghostty-web screenshot "$SESSION_WEB" --json

Native semantic renderer run:

SESSION_NATIVE=$(npx tsx src/cli/main.ts --home "$AGENT_TTY_HOME" --renderer libghostty-vt run --json -- npx tsx test/fixtures/apps/hello-prompt/main.ts | jq -r '.result.sessionId')
npx tsx src/cli/main.ts --home "$AGENT_TTY_HOME" --renderer libghostty-vt wait "$SESSION_NATIVE" --text "READY>" --json
npx tsx src/cli/main.ts --home "$AGENT_TTY_HOME" --renderer libghostty-vt type "$SESSION_NATIVE" "hello from libghostty-vt"
npx tsx src/cli/main.ts --home "$AGENT_TTY_HOME" --renderer libghostty-vt send-keys "$SESSION_NATIVE" Enter
npx tsx src/cli/main.ts --home "$AGENT_TTY_HOME" --renderer libghostty-vt wait "$SESSION_NATIVE" --text "hello from libghostty-vt" --json
npx tsx src/cli/main.ts --home "$AGENT_TTY_HOME" --renderer libghostty-vt snapshot "$SESSION_NATIVE" --json
npx tsx src/cli/main.ts --home "$AGENT_TTY_HOME" --renderer libghostty-vt screenshot "$SESSION_NATIVE" --json
npx tsx src/cli/main.ts --home "$AGENT_TTY_HOME" --renderer libghostty-vt record export "$SESSION_NATIVE" --format webm --out dogfood/20260424-libghostty-vt-renderer/videos/libghostty-vt-fallback.webm --json

Proof requirements:

• Save every JSON envelope from the commands above into the dogfood bundle.
• Copy/record the screenshot PNGs referenced by screenshot JSON into screenshots/.
• Capture a terminal recording of the dogfood session using asciinema rec if available, otherwise script.
• Attach at least one screenshot PNG and the WebM/asciicast proof in the final review handoff when running in Exec mode.
• Include native-info.json from @coder/libghostty-vt-node.getNativeInfo() so reviewers know exactly which native binding/Ghostty build was used.

Dogfood quality gate:

find dogfood/20260424-libghostty-vt-renderer -type f | sort
jq '.ok' dogfood/20260424-libghostty-vt-renderer/*.json
file dogfood/20260424-libghostty-vt-renderer/screenshots/*.png
file dogfood/20260424-libghostty-vt-renderer/videos/*.webm

Acceptance criteria

• ghostty-web remains the default renderer and existing tests continue to pass.
• --renderer ghostty-web and AGENT_TTY_RENDERER=ghostty-web behave like current default behavior.
• --renderer libghostty-vt uses @coder/libghostty-vt-node for snapshot, wait, and visible text.
• --renderer libghostty-vt screenshot ... succeeds by replaying into ghostty-web fallback for PNG rendering.
• record export --format webm continues to work when libghostty-vt is selected, either through hybrid fallback or explicit ghostty-web resolution.
• The native package is loaded lazily and only when libghostty-vt is selected.
• Unsupported/missing native addon cases produce clear actionable errors.
• Renderer selection works for both live-host RPC paths and offline replay paths.
• Config defaultRenderer and env var AGENT_TTY_RENDERER are documented and tested.
• Dogfood proof includes screenshots and at least one recording/video artifact.

Risks and mitigations

• Native package API/ABI instability: pin the dependency, record getNativeInfo() in dogfood, and keep mapping code isolated in src/renderer/libghosttyVt/.
• Semantic drift between ghostty-web and libghostty-vt: start with text/snapshot assertions, not pixel parity; screenshots remain ghostty-web.
• Running-host renderer overrides: avoid session-start-only env behavior by passing rendererName in render RPC params and by making HostRendererManager renderer-aware.
• Optional dependency/platform issues: lazy import, clear errors, and guarded native integration tests.
• Video capability mismatch: delegate video operations to ghostty-web and document that libghostty-vt is not a video renderer.